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ABSTRACT 


In this research, it is aimed to determine the success level of the students 
according to the answers given in the previous exams with TOPSIS method, to 
send the question to the related student by the authorized teacher according 
to the determined success level, to analyze the student's periodical (weekly- 
monthly-yearly) performance with graphical data and to create a data bank 
and to determine the overall performance of the student. Developmental 
research technique was used, which is one of the design-based research 
method derivatives including the design of educational tools. This study 
consists of question suggestion and question selection tool components. 
Adaptive question suggestion system perform generation, storage, 
classification, disclosure and publication of the content. The software has been 
successfully tested by 350 students and teachers by using 1000 questions 
prepared by the educator about the courses and subjects previously defined. 
As a result, it is seen that students check all the questions related to courses 
and subjects in the application database and submit the most appropriate 
question to the requestor in response to the question requests. Also it is 
observed that the TOPSIS algorithm quickly determined the desired number of 
questions from the question bank formed by the authorized teacher in 
accordance with the required criteria, and the process of creating a trial or 
subject test was performed successfully. 

KEYWORDS: adaptive question recommendation system; student achievement; 
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1. INRODUCTION 

Today, technology development and diversity related to 
education of technology is increasing. The most up-to-date 
topics are that students can access e-learning content not 
only at school but also at other times and participate in 
constructivist education that will enable the active 
participation of students (Kong et al., 2014) Adaptive 
Learning Process is a computer based education method that 
organizes and plans the education of each student in the 
most appropriate way considering various needs situations 
(Szijarto and Cousins, 2019).The question recommendation 
system is the process of evaluating the system for a purpose 
by means of automatic or manual tuning and trial process, 
and observing the number of answers given as expected 
because it meets the specified response rate, and presenting 
the results to the system by determining the differences 
between them (Hansen et al., 2019). In this study, a system 
has been developed including the functions of storing, 
classifying, modifying, searching and listing the questions 
using TOPSIS algorithm, creating, modifying and publishing 
new contents using the questions prepared again. 

2. MATERIALS AND METHODS 

In this paper, developmental research technique was used, 
which is one of the design-based research method 
derivatives including the design of educational tools. This 
study consists of question suggestion and question selection 
tool components. Adaptive question recommendation 
system perform generation, storage, classification, disclosure 
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and publication of the content. The main features of the 
question recommendation system are presented in Table 1. 
The developed software sends learning questions to 
students, answers them, and provides a level of achievement. 
Learning questions The most convenient and accurate way 
to mark the answers to test questions in SQL repositories 
can be provided by data standards. For this reason, some 
standardization studies were done on the data of learning 
questions (Stratakis et al., 2003). 


Tablel. The main features of the question 
_ recommendation system _ 


Feature 

Description 

Search or 
Find 

Be able to easily find Learning Tests 
with the data entered 

Quality 

control 

The Question Repository should 
provide a technical and pedagogical 
requirement comparison of Data and 
Test questions entered into the System 

Presentation 

process 

Test questions should be able to be sent 
to students' smart devices 

Addition 

Provide comprehensive search by 
gathering information from other 
question pools 

Request 

Question Repositories Must contain 
different options to provide test 
questions for learning 
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The developed software was developed with the aim of 
storing, searching, sharing, and selecting the appropriate test 
question for learning purposes. To accomplish these 
functions, an MSSQL Server 2016 database management 
system and LINQ to SQL and DataSet.xsd techniques can be 
used in conjunction with C # and MSSQL. These features are 
prepared with ASP.Net technology. This software, which 
processes according to the student achievement, allows the 
addition of learning objects for any different academic 
studies. 

3. ADAPTIVE QUESTION RECOMMENDATION SYSTEM 

Instructors use the Adaptive Question Recommendation 
System (AQRS] software to automatically send test question 
contents to the system of the course students, to make 
assessments about their study notes or students, and to 
access this recorded information at any time. In the software, 
relational database applications, WebApi applications and 
Another purpose of the software is to store learning objects 
and tests in the database and use them when needed. 

There are three types of user roles in AQRS software that are 
administrator, instructor and student. In order to add any 
objects or use other features of the software, users must 
register to the system. Membership requests are sent 
through the user of the Windows Form base. Dev Express 
[vl8.2.6] The Win Form library functions to interface the 
software, and then administrators examine these requests to 
allow appropriate user roles. After the administrator has 
entered the usage role by the administrator, users can use 
the AQRS software. Learning tests and objects are stored in 
the system via MSSQL Server interfaces. If necessary, 
learning can be easily accessed by testing and listing method 
by data or types of objects. 

In the AQRS software, the necessary components for the 
learning test and object data were prepared. In addition, any 
data format can be stored in the system as a JSON-based file 
format and can be converted to Web Api format at any time 
and sent. Users can answer, analyze, comment on learning 
objects and tests, and identify problems using their 
interfaces. One of the most important features of the AQRS 
software is that the instructor approves the quality control 
of test questions and objects in the system. After checking 
the test questions prepared for learning purposes, the test 
questions have the feature that all students registered in the 
software can be transmitted to their mobile devices. The 
main page of the authorized instructor and the unauthorized 
instructor for access to the system shown in Figure 1 is as 
follows. 
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Figurel. Instructor and the unauthorized instructor 
for access to the system 

3.1. Architecture of AQRS software 

The developed AQRS software is based on a multi-layered 
software architecture and an object-oriented programming 
approach. There are two horizontal layers in the upper layer 
architecture of the software. These are both the presentation 
layer and the data layer. These layers have the component 
structures required for the system operational state of the 
AQRS software shown in Figure 2. The existence of these two 
layers makes the software project modular. For example, 
extra component structures can be added to the software 
simply and quickly. The connection between the database 
and the application in the data layer is prepared. 

The Entity Framework sets an open source object option for 
theADO.NET andC # .NET programming language. In Figure 
2, instructors can use these interfaces to add, list, and view 
the data of learning tests using the Windows Form base. In 
the work layer, there are operations of the data conversion 
engine. When a request is received by the presentation layer, 
the system makes a decision and transmits it to the data 
layer in order to process the workflow state that appears in 
the software faster. The data layer adds comments about the 
instructor and students, reporting problems, users and roles, 
and all data such as the topic and the sub-subject that fits the 
database. 
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Figure2. Architectural structure of AQRS 


3.2. Selection of Tests with TOPSIS Algorithm 

Trainers and students try to find the most appropriate 
learning objects from many sources (Sinclair et al., 2013], 
For this reason, it is necessary to carry out efficient studies 
on the quality of tests for learning purposes. The quality of 
learning objects or tests is difficult to measure because 
instructors and learners can respond according to the 
requirements and requirements to which they depend. 
There are many selection methods to find the test that is 
most suitable for learning. There are many types of software 
to measure the quality status of tests for learning purposes 
(Vargo et al., 2003], 

The AQRS application is set to submit test questions by 
evaluating the status of measurement and analysis according 
to the achievement status, interaction, participation, 
availability and content categories of the students with 
TOPSIS algorithm. According to the scoring results, there are 
various softwares that use multi-criteria evaluation methods 
to select the best alternative test for learning (Kurilovas, 
2009], There are cases where each multi-criteria decision 
making technique is superior to other criterion techniques 
(Omiirbek et al., 2013], For this reason, which multi-criteria 
decision-making method should be applied to the decision 
problem, natural structure of decision problem, various 
options, scales of evaluations, type status of uncertain rate, 
dependency ratio of criteria, expectation status of the 
decision maker, data quantity and quality status to be 
entered factors should also be taken into consideration 
(Ceiling and Hatami-Marbini, 2011], 


TOPSIS algorithm is applied in production, marketing, 
planning, group decision making, distribution of resources, 
education, health, selection of markets, transportation, 
support of decisions and many other areas (Geuna and 
Martin,2003], The TOPSIS algorithm is known as one of the 
most excellent multi-criteria decision making methods 
because it has the advantage of being able to sort out the the 
excessively large number of alternatives and to quickly 
determine the best alternative outcome (Tavana and 
fiatami-Marbini, 2011], 


TOPSIS algorithm is used in AQRS software according to 
student achievement level. This decision-making criterion is 
based on the theory of priority. Also, problems involving 
complex decision-making criteria can be easily solved with 
TOPSIS. It is useful for logically using the data entered into 
the system and the decisions of experts. Since the TOPSIS 
algorithm does not contain mixed mathematical expressions, 
it is easy to use and has a transparent structure for decision 
makers because of the sorting criteria. The steps for 
selecting adaptive questions with TOPSIS are shown as 
follows. 

In the TOPSIS method, the state of formation of the decision 
matrix (A], the state of formation of the standard decision 
matrix (Rj, the state of formation of the weighted standard 
decision matrix (V], the states of formation of the solutions 
in the ideal direction CyT) and the negative ideal direction 
(j 4“). It consists of a number of steps, such as calculating the 
degree of discrimination criteria and calculating the relative 
proximity to the ideal solution. 

Step 1: First, the decision matrix (A] needs to be established. 
The evaluation matrix is started by creating the decision 
matrix in which alternative values are required to be listed 
in the rows and criteria to be evaluated based on the 
decision-making evaluation in the columns. The step by the 
decision makers, known as the initial matrix, which consists 
of alternative and n criterion evaluations, is as follows: 



H n ■■■ 

H 1J ■“ 


ij ~ 

°21 ... 

Hjjf ... 

°s.n 


Hpy ... 

h. 


Step 2: The standard decision matrix (Rj needs to be 
established. The initial values that take place in our decision 
matrix must be brought to the standard state. 

Step 3: Since the weighted standard decision matrix (V] 
needs to be established, the weighting ratios fwjof the 
criterion assessments must first be determined. 

2>=i 

L=1 

Then, each individual value found in the R matrix must be 
multiplied by the values of the criterion to which it is 
connected and a Weighted Standard Decision Matrix (V] 
should be created. 

Step 4: Solution results in the ideal direction (A 4 ) and 
negative in the ideal direction C/T) should be established. 
The TOPSIS method assumes that each criterion assessment 
has shown a tendency to increase or decrease steadily. For 
this reason, the maximum value in each column in the matrix 
V represents the ideal direction (A*) solution, the minimum 
value is the negative ideal solution CO solution. 

rf = ‘ = i 

= t= 1, 
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Step 5: The state of calculation of the separation criteria 
degrees. The distance in the Euclidian Approach is used to 
calculate the distance of the criterion at each decision point 
to the solution point in the ideal direction and the negative 
ideal direction. According to these values, the distance of the 
ideal solution j* in the ideal solution and the distance of the 
ideal solution in the negative direction J L "are calculated by 
the formulas shown below. 



The determination of the criteria for selecting the test 
question for learning purposes and other criteria belong to 
the administrator or trainer and the criteria are adjusted 
according to the test statistics. These criteria can be 
evaluated manually by instructors by manual input or by 
decision-making methods. 

The trainers were interviewed to determine the selection 
criteria of the data-based test questions. Ten instructors 
participated in the interview. According to the data in the 
interview, TOPSIS question coefficient criteria (reliability, 
discrimination, cognitive area, degree of difficulty of the 
question] were determined for the selection of test 
questions. 



Step 6: Calculate the degree of proximity relative to the ideal 
solution result. Using the distinction criteria in the ideal 
direction and in the negative ideal direction, the relative 
proximity of each alternative case to the ideal solution result 
C‘ is calculated. 

C- = c l 6 [1U] 1 . m 


Test questions with the highest value obtained by TOPSIS 
algorithm are selected. As shown in Table 2, the AQRS 
software was tested with four criteria. In the first few trials, 
the CRs of the criteria were found to be greater than 0.2, 
indicating that the criteria did not proceed consistently with 
the selection of test questions. At the end of the fifth trial, the 
CR of the criteria was found to be 0.877802262, which was 
below 0.877, and the criteria and points given were 
therefore consistent. When necessary, instructors can 
change the number of criteria up to 10 or even more and 
enter alternatives according to preference. Test questions 
are evaluated according to four different criteria. 


Table2. Criteria weights 


Question 

Number 

Criteria weights 

Cl 

C2 

C3 

C4 

Total 

S- 



0,1 

0,3 

0,4 

0,2 


Q14 

6 

3 

5 

4 

0,877802262 

0,012309 

0,088422 

Q8 

5 

3 

4 

3 

0,719027294 

0,028831 

0,073780 

Q9 

5 

3 

4 

3 

0,719027294 

0,028831 

0,073780 

Q13 

4 

2 

5 

4 

0,711699469 

0,030011 

0,074085 


4. CONCLUSION 

In this research, it is aimed to determine the success level of 
the students according to the answers given in the previous 
exams with TOPSIS method, to send the question to the 
related student by the authorized teacher according to the 
determined success level, to analyze the student's periodical 
(weekly-monthly-yearly] performance with graphical data 
and to create a data bank and to determine the overall 
performance of the student. The software has been 
successfully tested by 350 students and teachers by using 
1000 questions prepared by the educator about the courses 
and subjects previously defined. As a result, it is seen that 
students check all the questions related to courses and 
subjects in the application database and submit the most 
appropriate question to the requestor in response to the 
question requests. Also it is observed that the TOPSIS 
algorithm quickly determined the desired number of 
questions from the question bank formed by the authorized 
teacher in accordance with the required criteria, and the 
process of creating a trial or subject test was performed 
successfully. 
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